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ABSTRACT 

Since 1994, the National Science Foundation has funded Rural 
Systemic Initiatives (RSI), a set of systemic reforms to enhance math, 
science, and technology education in economically disadvantaged, rural areas 
through instructional and policy reform and community development activities. 
As part of an evaluation of the program, surveys were sent to each school 
district in the Delta RSI; the Appalachian RSI; and the UCAN RSI, which 
consisted of Utah, Colorado, Arizona, and Nevada. Findings indicate that 
elementary schools had the highest level of participation in RSIs. RSIs 
employed one of two different strategies: provision of a very broad menu of 
services to everyone, or provision of core services to everyone and 
supplementation with other services in certain schools. Professional 
development was listed as the most effective RSI activity. Curriculum-related 
activities were also reported to be among the most effective RSI activities. 
Factors that facilitated education reform were administrative support and 
state education standards. Barriers to reform were lack of money and lab 
equipment and low expectations for students. RSI participation most influenced 
math assessment alignment with standards; emphasis on strengthening science, 
math, and technology; and curriculum implementation. RSI least influenced 
teacher-student ratios, minority test scores, and the percentage of students 
taking advanced placement courses. Some of the most effective benefits 
provided by RSI were equipment/materials, consultants, and funding. Activities 
that led to school staff going beyond the confines of their buildings were 
beneficial because they led to increased communication with the community and 
other schools. (Contains 24 data tables.) (TD) 
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Rural Systemic Initiatives Survey 

In 1 994, funding of Rural Systemic Initiatives (RSI), the third in a set of systemic reform 
initiatives sponsored by the National Science Foundation (NSF), was offered on a competitive 
basis. The RSIs were created to enhance math, science, and technology education for all 
students and focus on economically disadvantaged, rural areas through instructional and policy 
reform and community development activities. RSIs are expected to address policy, leadership, 
and work force issues related to educational barriers by engaging community involvement in 
creating a comprehensive and sustainable system of math, science, and technology education that 
reflects current advancements in these areas. Instruction and performance standards are 
expected to play a major role for districts involved in these programs. 

In August 1998, NSF contracted with The Evaluation Center to evaluate the RSI program. In 
part, the study was designed to investigate the relationship among the potentially conflicting 
values in traditional rural communities with the current understanding of education reform and 
the goals of NSF-sponsored Rural Systemic Initiatives, including the six drivers of educational 
system reform. More importantly, the Center focused on the impact of these relationships on 
student learning in science and mathematics within selected RSI projects. The RSIs to be 
included in the study were the Delta RSI (Mississippi, Arkansas, and Louisiana), the 
Appalachian RSI (Ohio, West Virginia, Virginia, Kentucky, Tennessee, North Carolina) and the 
UCAN RSI (Utah, Colorado, Arizona, and Nevada). 

In order to achieve its objectives, a mixed method approach was used for the study. Using the 
User-Friendly Handbook for Mixed Method Evaluations distributed by NSF as a basic set of 
guidelines, broad-based sets of sets of quantitative and qualitative data were be collected and 
analyzed. One of the data sets was from a survey of stakeholders in school districts participating 
in RSIs being studied by The Evaluation Center. The purpose of the survey was to help 
evaluators identify local conditions that might influence the extent to which the “drivers of 
educational reform” had been implemented in the district. 

In April 2000, a paper and pencil questionnaire was sent to a matrix sample of stakeholders (the 
designated contact persons for each school district participating in the RSIs). The first five 
pages of the questionnaire were identical. However, on the last page, each stakeholder received 
one of three different subsets of questions. After about a month, a second questionnaire was sent 
to nonrespondents. The final response rates for the RSIs being studied were Delta=31 percent; 
UCAN=33 percent; ARSI=51 percent. Data were analyzed using quantitative and qualitative 
methods. The findings from the survey are found below as summarized data and an 
interpretation of the response for each question. 
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Question 1 — Circle the RSI in which your district participated 

Even though ARSI had the highest response rate (5 1 percent), the respondents from UCAN 
schools make up the greatest percentage of the total responses (38.8 percent). Delta had the 
lowest response rate (31 percent) and the lowest percentage of total responses (26.9 percent). 



Table 1. Frequency and Percentage of Respondents to the Survey 



RSI 


Frequency 


Percent 


Delta 


18 


26.9% 


UCAN 


26 


38.8% 


ARSI 


23 


34.3% 


Total 


67 


100% 



Question 2— List the names and levels of schools in your district that have participated in 
some substantial way with this RSI. 

Elementary school participation in the RSIs was higher than high school participation, while 
middle school/junior high participation varied. In all three RSIs, twice as many elementary 
schools as high schools were reported to have substantial involvement with the RSI (see Table 
2). Involvement at the middle school/junior high level was similar to the high school level for 
ARSI, similar to the elementary level for Delta, and intermediate for UCAN. 

Table 2. Number of Participating Schools in Each RSI at Three Educational Levels 
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Number of Participating Schools 


RSI 


Elementary 


Middle/Jr High 


High School 


Delta 


35 


32 


18 


UCAN 


46 


25 


17 


ARSI 


52 


19 


23 


Total 


133 


76 


58 



These differences in the number of schools involved at each level could be due to the fact that 
districts frequently have more elementary schools than middle schools/junior highs and more 
middle level schools than high schools. However, in each RSI a higher percentage of districts 
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reported participation at the elementary level than at the high school level (see Table 3). Middle 
school participation varied in relation to the lower and upper levels among the three projects. 

Table 3. Percentage of Districts in Each RSI Reporting Participation at Three Educational 
Levels 
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Percentage of Districts 


RSI 


Elementary 


Middle/Jr High 


High School 


Delta 


72% 


94% 


61% 


UCAN 


81% 


77% 


65% 


ARSI 


74% 


61% 


70% 


Total 


76% 


77% 


65% 



Assignment of schools to grade levels - Some respondents listed grade levels for each school 
while others only listed elementary, middle school/junior high, or high school. Schools with 
word designations were assigned to those categories. Schools with grade level information were 
assigned to each of the three categories based on the major focus of the school. “Elementary” 
includes schools with grade ranges of prekindergarten or kindergarten through grades 5, 6, or 7. 
“Middle School/Junior High” includes schools with grade ranges of 5, 6, or 7 through grades 8 or 
9. “High School” includes grades 9 or 10 though 12. Schools that serve grades K-8 were 
counted as both elementary and middle school; schools serving grades 7-12 were counted as 
both middle school and high school. 

Question 3— School district information 

The data in Table 4 reveal that there is a substantial amount of variation on key characteristics 
among rural districts participating in RSI. The standard deviations for many of the district 
characteristics are very high— in some cases as high as the mean. This finding shows that there is 
no such thing as a “typical” rural district. 

Table 4. District Characteristics 



^ IIIIIII 


Delta 


UCAN 


ARSI 


Total 





Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Total schools in district 


6 


4 


4 


3 


9 


5 


6 


4 


Number of teachers 


177 


107 


73 


74 


264 


143 


168 


137 
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Delta 


UCAN 


ARSI 


Total 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Number of students 


2772 


1491 


1052 


1146 


3490 


1904 


2391 


1859 


Percent free lunch 


75.62 


13.63 


79.04 


19.57 


63.48 


13.4 


72.59 


17.24 


Percent minority 


64.66 


26.62 


70.44 


31.99 


2.49 


4.66 


45.75 


39.41 


Percent high school graduate 


74.04 


21.92 


83.55 


19.91 


84.65 


11.16 


81.2 


18.2 


Percent of grads attending 
college 


39.56 


12.71 


42.78 


24.31 


46.67 


14.26 


43.26 


17.94 


Percent postsecondary 
education 


16.29 


6.73 


18.38 


15.95 


26.52 


24.95 


20.95 


18.67 


Annual per-pupil 
expenditure 


4904 


2067 


4075 


1904 


5147 


1619 


4706 


1882 


District’s geographic size 
(sq. mi.) 


449 


228 


3714 


12353 


393 


138 


1518 


7155 


Population of largest town in 
area 


4933 


4922 


6365 


15966 


3764 


3513 


5048 


10078 


Distance from city of 
100,000+ 


85 


40 


125 


93 


102 


55 


106 


70 


Yrs. of involvement with 
RSI 


2.61 


1.24 


4.38 


1.88 


3.17 


1.15 


3.46 


1.64 



Question 4— Which of the following RSI activities or services has your school district 
participated in or received over the past 5 years? (Check all that apply) 

Examination of the data in Table 5 suggests two different strategies employed by RSIs for 
providing services to school districts. ARSI apparently tried to provide a very broad menu of 
services to everyone. UCAN and Delta appear to provide core services, such as group/individual 
professional development and on-site consultation, to everyone and supplement with other 
services, perhaps tailored to the needs of individual districts. 



Table 5. RSI Activities or Services Over Past Five Years 



Delta 


UCAN 


ARSI 


Total 




5 





Freq 


%* 


Freq 


%* 


Freq 


%* 


Freq 


%* 


Group professional development 


16 


88.9 


26 


100 


22 


95.7 


64 


95.5 


Individual professional develop. 


10 


55.6 


19 


73.1 


21 


91.3 


50 


74.6 


On-site consultation 


15 


83.3 


22 


84.6 


22 


95.7 


59 


88.1 


Needs assessment 


9 


50.0 


18 


69.2 


21 


91.3 


48 


71.6 


Curriculum audit 


5 


27.8 


17 


65.4 


17 


73.9 


39 


58.2 


Align curriculum w/ standards 


8 


44.4 


21 


80.8 


18 


78.3 


47 


70.1 


Develop curriculum 


4 


22.2 


16 


61.5 


16 


69.6 


36 


53.7 


Materials selection 


6 


33.3 


15 


57.7 


19 


82.6 


40 


59.7 


Instructional skills 


9 


50.0 


17 


65.4 


20 


87.0 


46 


68.7 


Student assessments 


10 


55.6 


15 


57.7 


14 


60.9 


39 


58.2 


Provide materials 


7 


38.9 


14 


53.8 


17 


73.9 


38 


56.7 


Community involvement 


9 


50.0 


9 


34.6 


17 


73.9 


35 


52.2 


Proposal development 


4 


22.2 


8 


30.8 


14 


60.9 


26 


38.8 


Other 


2 


11.1 


2 


7.7 


1 


4.3 


5 


7.5 


Other 


1 


5.6 










1 


1.5 



Note: * indicates percent of cases. 

Question 5— Briefly describe any particularly effective or exemplary RSI-sponsored or 
RSI-initiated activities that have taken place in your school district. 

The most effective RSI activities reported by districts focused on professional development for 
teachers, curriculum changes, increased resources, and increased communication and 
involvement (see Table 6). 




7 



6 



Table 6. Most Effective or Exemplary RSI-Sponsored or RSI-Initiated Activities 
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Number of Districts 


Percentage of Districts 


In 


Activity 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Overall 


Professional development 


12 


18 


15 


67% 


69% 


65% 


67% 


Curriculum activities 


6 


10 


11 


33% 


39% 


48% 


40% 


Resources 


5 


5 


7 


28% 


19% 


30% 


24% 


Communication/network 


5 


4 


6 


28% 


8% 


26% 


22% 



Professional development was listed most frequently as a particularly effective or exemplary 
RSI-sponsored activity, with 67 percent of the districts reporting one or more items in this 
category (see Tables 6 and 7). The Teacher Partner program received high praise from 39 
percent of the districts in ARSI. These teachers not only participated in regular professional 
development meetings themselves, but also trained other teachers in their schools. Several ARSI 
districts also mentioned professional development focused on inquiry. Districts in UCAN 
reported professional development activities focused on interpretation of standards, aligning 
curriculum with standards, assessment, instructional skills and techniques, training to use new 
curriculum, and leadership development as most effective. One third of the Delta respondents 
listed training to use new curriculum as the most effective RSI activity. Several respondents also 
mentioned training in instructional skills and techniques and training for administrators. More 
than 25 percent of the respondents in UCAN and ARSI, as well as several Delta districts, listed 
professional development for teachers, with no topics specified, as the most effective RSI- 
sponsored activity. While most professional development was directed toward teachers, 
activities also were provided for administrators, counselors, and leadership teams. 

Table 7. Professional Development Activities Listed as Effective by Two or More Respondents 





Number of Districts 


Percentage of Districts 


Professional Development Activity 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Interpretation of standards 


1 


5 


0 


6% 


19% 


0% 


Aligning curriculum with standards 


1 


4 


2 


6% 


15% 


9% 


Training to use new curriculum 


6 


3 


0 


33% 


12% 


0% 


Instructional skills and techniques 


2 


3 


2 


11% 


12% 


9% 


Inquiry 


0 


0 


4 


0% 


0% 


17% 
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Number of Districts 


Percentage of Districts 


Professional Development Activity 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Assessment 


0 


4 


0 


0% 


15% 


0% 


Training for new Gateway tests 


0 


0 


2 


0% 


0% 


9% 


Training to use new equipment 


1 


0 


2 


6% 


0% 


9% 


Teacher Partner program 


1 


0 


9 


6% 


0% 


39% 


Professional development (no topic 
identified) 


2 


7 


6 


11% 


27% 


26% 


Leadership development 


0 


4 


1 


0% 


15% 


9% 


Administration training 


2 


0 


1 


11% 


0% 


9% 



RSI activities related to curriculum changes were also listed as particularly effective by many 
districts (see Tables 6 and 8). Several districts in Delta and UCAN listed assistance aligning 
their curriculum with standards and with selecting, developing, and implementing new 
curriculum materials as helpful RSI activities. Almost a third of ARSI districts reported 
assistance with curriculum audits as particularly useful. 

Table 8. Curriculum-Related Activities Listed as Effective by Two or More Respondents 





Number of Districts 


Percentage of Districts 


Activity 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Curriculum alignment with standards 


3 


6 


2 


17% 


23% 


9% 


Curriculum audit 


0 


2 


7 


0% 


8% 


30% 


Needs assessment/program review 


1 


0 


1 


6% 


0% 


4% 


Develop action plan 


1 


2 


1 


6% 


8% 


4% 


Assistance with selection, 
development, and implementation of 
curriculum materials 


2 


4 


2 


11% 


15% 


9% 



Increased resources (both external funding and the things that external funding can buy) was a 
third type of RSI activity that respondents felt was particularly effective (see Tables 6 and 9). 
Districts in all three RSIs reported increased funding (unspecified) as helpful. Several Delta 
districts also listed assistance from consultants and specialists. ARSI districts mentioned 
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receiving new equipment and associated training for teachers. 

Table 9. Resources Listed as Effective by Two or More Respondents 
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Number of Districts 


Percentage of Districts 


Resource 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Funding 


4 


3 


1 


22% 


12% 


4% 


Grant-writing assistance and info 


0 


0 


2 


0% 


0% 


9% 


Consultants and specialists 


3 


0 


2 


17% 


0% 


9% 


Equipment 


0 


0 


3 


0% 


0% 


13% 


Resources (video, etc.) 


0 


2 


1 


0% 


8% 


4% 



RSI-sponsored activities that led to increased communication and involvement were also listed 
as particularly effective (see Tables 6 and 10). Districts in Delta and ARSI were enthusiastic 
about the increase in parental and community involvement, particularly as a result of Family 
Math and Science nights. Several UCAN districts developed increased communication with 
Native American tribes and with other districts serving Indian students. As a result of this effort, 
three tribes and four districts formed the UTE Nations Education Coalition, which is applying 
for 501(c)(3) status. 

Table 10. Increased Communication and Community Involvement as a Result of RSI Activities 





Number of Districts 


Percentage of Districts 


Activity 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Parental/community involvement 


5 


0 


5 


28% 


0% 


22% 


Communication with tribes and 
districts 


0 


3 


0 


0% 


12% 


0% 


Networking among teachers 


0 


1 


1 


0% 


4% 


9% 



Overall, the respondents reported professional development for teachers as the most effective 
RSI-sponsored activity. Other effective activities varied among the three programs. Delta 
districts found professional development for teachers (67 percent), particularly training to use 
new curriculum (33 percent), most useful as well as increased parental and community 
involvement (28 percent) and increased funding (22 percent). UCAN respondents listed 
professional development for teachers (69 percent) and assistance aligning curriculum with 
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standards (23 percent) as most helpful. ARSI districts found professional development for 
teachers (70 percent), particularly the Teacher Partner program (39 percent), most useful, as well 
as curriculum audits (30 percent) and increased parental and community involvement (22 
percent). 

Question 6— What is the greatest benefit your school district has gained from involvement 
with the RSI? 

The greatest benefits reported by districts focused on professional development for teachers, 
increased resources, curriculum changes, and increased communication and networking (see 
Table 11). 



Table 11. Greatest Benefit to School District From RSI Involvement 
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Number of Districts 


Percentage of Districts 




Benefit 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Overall 


Professional development 


10 


13 


11 


56% 


50% 


48% 


51% 


Resources 


7 


9 


12 


39% 


35% 


52% 


42% 


Curriculum changes 


2 


12 


7 


11% 


46% 


30% 


31% 


Increased involvement 


3 


9 


6 


17% 


35% 


26% 


27% 



Professional development was listed by 51 percent of the districts as the greatest benefit they 
derived from RSI involvement (see Tables 1 1 and 12). While most of the professional 
development options listed on the survey were directed toward teachers, several districts also 
listed professional development for administrators and leadership training as benefits. 

Table 12. Targets of Professional Development Activities 
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Number of Districts 


Percentage of Districts 


Professional development targets 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Teachers 


8 


12 


11 


44% 


45% 


48% 


Leadership training 


0 


2 


1 


0% 


8% 


4% 


Administrators 


2 


1 


0 


11% 


4% 


0% 



Increased resources were listed by 42 percent of the respondents as a benefit of RSI involvement 
(see Tables 1 1 and 13). In addition to unspecified resources, these resources included 
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information, funding, consultants and contact with experts, as well as equipment, kits, and 
science/math materials. 



Table 13. Resources Listed as a Benefit of RSI Involvement 
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Number of Districts 


Percentage of Districts 


Type of resource 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Information 


1 


4 


4 


17% 


15% 


17% 


Funding 


4 


4 


0 


22% 


15% 


6% 


Equipment/kits/materials 


1 


2 


4 


6% 


8% 


17% 


Consultants and contact with experts 


3 


1 


2 


17% 


4% 


9% 


Unspecified 


0 


3 


5 


0% 


15% 


22% 



Curriculum changes were listed as a benefit of RSI involvement by 3 1 percent of the districts 
(see Tables 1 1 and 14). These changes included curriculum review and alignment with 
standards, curriculum selection and implementation, increased use of standards, and the 
development of a standards-based curriculum. 

Table 14. Curriculum Changes Listed as a Benefit of RSI Involvement 





Number of Districts 


Percentage of Districts 


Curriculum Change 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Alignment with standards 


1 


7 


5 


6% 


27% 


22% 


Use of standards 


0 


1 


3 


0% 


4% 


13% 


Standards-based curriculum 


0 


2 


1 


0% 


8% 


4% 


Selection and implementation 


1 


5 


0 


6% 


19% 


0% 



For 27 percent of the respondents, RSI involvement led to increased involvement with others 
outside the school (see Tables 1 1 and 15). This included increased communication and 
networking with colleagues, increased communication with community groups such as tribes, 
and increased family involvement. 
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Table 15. Increased Involvement with Others Outside the School as a Benefit of RSI 
Involvement 





Number of Districts 


Percentage of Districts 


Type of Involvement 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Communication 


0 


4 


1 


0% 


15% 


6% 


Networking and collaboration 


2 


5 


3 


11% 


19% 


13% 


Family involvement 


1 


0 


2 


6 % 


0% 


9% 



In addition to the benefits of RSI involvement listed by respondents in all three RSIs, several 
benefits were listed primarily by districts in ARSI. These included an increased focus on science 
and math and an increased motivation to pursue a standards-based curriculum (30 percent), 
improved instruction (26 percent), and the Teacher Partners (22 percent). 

Question 7— What factors have facilitated or been a barrier to education reform in your 
school district? (Check all that apply). 

Table 16 contains data on facilitating factors. The most important factor that was found to 
facilitate education reform across all three RSIs was administrative support from school boards, 
district administration, and school principals. Another factor that was found to facilitate 
education reform was state education standards. 



Table 16. Factors that Facilitated Education Reform 



, : 


Delta 


UCAN 


ARSI 


Total 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


District administration 


17 


94.4 


20 ' 


76.9 


22 


95.7 


59 


88.1 


School principals 


16 


88.9 


23 


88.5 


20 


87.0 


59 


88.1 


Personnel changes 


7 


38.9 


6 


23.1 


8 


34.8 


21 


31.3 


Teacher preparation 


10 


55.6 


12 


46.2 


10 


43.5 


32 


47.8 


Teacher subject knowledge 


9 


50.0 


15 


57.7 


8 


34.8 


32 


47.8 
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Delta 


UCAN 


ARSI 


Total 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


Teacher turnover 


2 


11.1 


2 


7.7 


4 


17.4 


8 


11.9 


Facilities 


8 


44.4 


5 


19.2 


9 


39.1 


22 


32.8 


Money 


4 


22.2 


5 


19.2 


4 


17.4 


13 


19.4 


Lab equipment 


3 


16.7 


10 


38.5 


4 


17.4 


17 


25.4 


Science materials 


7 


38.9 


12 


46.2 


4 


17.4 


23 


34.3 


Math materials 


12 


66.7 


14 


53.8 


10 


43.5 


36 


53.7 


Computers available 


12 


66.7 


12 


46.2 


16 


69.6 


40 


59.7 


Technology available 


10 


55.6 


8 


30.8 


12 


52.2 


30 


44.8 


Educational materials 


11 


61.1 


12 


46.2 


12 


52.2 


35 


52.2 


District projects 


10 


55.6 


9 


34.6 


16 


69.6 


35 


52.2 


State standards 


14 


77.8 


22 


84.6 


20 


87.0 


56 


83.6 


School board 


15 


83.3 


15 


57.7 


19 


82.6 


49 


73.1 


Parental involvement 


8 


44.4 


8 


30.8 


8 


34.8 


24 


35.8 


Community involvement 


7 


38.9 


8 


30.8 


9 


39.1 


24 


35.8 


Expectations for students 


8 


44.4 


8 


30.8 


8 


34.8 


24 


35.8 


Community support 


7 


38.9 


7 


26.9 


6 


. 26.1 


20 


29.9 


Court decisions 


2 


11.1 


3 


11.5 


3 


13.0 


8 


11.9 


School consolidation 


2 


11.1 


2 


7.7 


3 


13.0 


7 


10.4 


Other 


1 


5.6 










1 


1.5 



Note: * indicates percent of cases 

Table 17 contains data on barriers to education reform. The factors that were found to be barriers 
to education reform were money and lab equipment. The barriers were of lower magnitude than 
the facilitating factors. For example, in Table 17, the highest frequency was 45 (68.2 percent). 

In Table 16, the highest frequency was 59 (88.1 percent). School consolidations and school 
boards were not seen as barriers to education reform. 
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Table 17. Factors That Were a Barrier to Education Reform 





Delta 


UCAN 


ARSI 


Total 




Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


District administration 


1 


5.6 


5 


19.2 


1 


4.5 


7 


10.6 


School principals 


2 


1 L 1 


3 


11.5 


5 


22.7 


10 


15.2 


Personnel changes 


7 


38.9 


12 


46.2 


7 


31.8 


26 


39.4 


Teacher preparation 


7 


38.9 


13 


50.0 


5 


22.7 


25 


37.9 


Teacher subject knowledge 


6 


33.3 


9 


34.6 


8 


36.4 


23 


34.8 


Teacher turnover 


14 


77.8 


8 


30.8 


10 


45.5 


32 


48.5 


Facilities 


6 


333 


9 


34.6 


7 


31.8 


22 


33.3 


Money 


14 


77.8 


17 


65.4 


14 


63.6 


45 


68.2 


Lab equipment 


14 


77.8 


10 


38.5 


19 


86.4 


43 


65.2 


Science materials 


9 


50.0 


12 


46.2 


16 


72.7 


37 


56.1 


Math materials 


5 


27.8 


8 


30.8 


11 


50.0 


24 


36.4 


Computers available 


5 


27.8 


8 


30.8 


5 


22.7 


18 


27.3 


Technology available 


6 


33.3 


9 


34.6 


8 


36.4 


23 


34.8 


Educational materials 


4 


22.2 


4 


15.4 


3 


13.6 


11 


16.7 


District projects 


2 


11.1 


6 


23.1 


2 


9.1 


10 


15.2 


State standards 


2 


11.1 


1 


3.8 






3 


4.5 


School board 






2 


7.7 


1 


4.5 


3 


4.5 


Parental involvement 


8 


44.4 


7 


26.9 


9 


40.9 


24 


36.4 


Community involvement 


8 


44.4 


8 


30.8 


6 


27.3 


22 


33.3 


Expectations for students 


9 


50.0 


7 


26.9 


11 


50.0 


27 


40.9 




15 



14 





Delta 


UCAN 


ARSI 


Total 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


Freq. 


%* 


Community support 


11 


61.1 


9 


34.6 


9 


40.9 


29 


43.9 


Court decisions 






3 


11.5 


5 


22.7 


8 


12.1 


School consolidation 


1 


5.6 






1 


4.5 


2 


3.0 


Other 1 


2 


11.1 


2 


7.7 


2 


9.1 


6 


9.1 


Other 2 






2 


7.7 


2 


9.1 


4 


6.1 



Note: * indicates percent of cases 



Examination of Tables 16 and 17, together, enables one to detect differences among the RSIs. 
Leadership, teachers, standards, materials, computers/technology, and district projects facilitated 
education reform for Delta. The challenge Delta faces is teacher turnover, lab equipment, 
money, expectations for students, and community support. Leadership, teachers, standards, and 
materials facilitated education reform for UCAN. The challenge UCAN faces is teacher 
preparation and money. Leadership, standards, computer availability and district projects 
facilitated education reform for ARSI. The challenges ARSI faces are money, lab equipment, 
materials, and expectations for students. 

Question 8— What effect has participation in the RSI (UCAN, ARSI, or Delta) had in your 
school district on the following indicators related to the “drivers of educational reform?” 

On question 8, each respondent received one of three different subsets of items. The scale of the 
Likert-like items was 1= unaffected; 2=somewhat increased; 3 positively increased; and 
4=greatly increased. The aggregated descriptive statistics are found in Table 18. Respondents 
said RSI participation most influenced math assessment alignment with standards (mean=3.58); 
changing of math activities to improve curriculum (3.52); and lowest ability students’ 
achievements in science and math (3.50). RSI least influenced the percentage of students taking 
Advanced Placement college examinations (mean=l .67); low income parents’ support of their 
children’s learning in science and math (1.80); and minority students’ scores on science and math 
standardized tests (1.85). Because of the matrix sampling, the Ns for each item were lower than 
they would have been otherwise. Also, when variables are considered on a listwise basis, there 
are none available for analysis. This precluded the use of more advanced statistical analysis. 

Table 18. Result of RSI Participation 



Indicators 


N 


Mean 


SD 


Secondary math curriculum alignment with the National Council 


16 


2.50 


.89 
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Indicators 


N 


Mean 


SD 


of Teachers of Mathematics standards 








Teaching of science relevant to our locale 


16 


2.44 


.96 


Teachers’ abilities to articulate instructional standards 


16 


2.69 


.87 


Teachers’ and staffs participation in professional development 
programs 


16 


3.00 


.97 


Policies for providing a coherent vision that encompasses all 
students 


16 


2.50 


.97 


Teacher training in the use of assessment 


16 


2.87 


.81 


Acquisition of additional funds to support science and math 
education 


16 


2.63 


.89 


Facilities for providing access to learning technologies 


15 


2.47 


1.19 


Low income parents’ support of their children’s learning in science 
and math 


15 


1.80 


.56 


Relevant stakeholders’ understanding and acceptance of 
systematic change as a strategy for improving education 


16 


2.44 


.96 


Number of middle school/jr. high students engaged in challenging, 
high quality science and math learning experiences 


16 


2.88 


1.82 


High school students’ interest in math and science 


16 


2.13 


.72 


Use of valid and reliable student performance measures 


16 


2.56 


.81 


Percentage of historically underserved students showing success 
in math and science courses 


16 


2.50 


1.79 


Lowest ability students’ achievements in science and math 


16 


3.50 


2.83 


Science curriculum alignment with our state’s science curriculum 
standards 


25 


2.88 


.93 


Teaching of math that reaches all students 


24 


2.54 


.88 


Teachers’ use of hands-on, inquiry-based science instruction 


25 


2.76 


.78 


Adequate time and support for teachers’ ongoing professional 
development 


25 


2.40 


.96 


Enrollment of all students in high quality, rigorous programs 


23 


1.91 


.90 
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Indicators 


N 


Mean 


SD 


Provision of professional development that focuses on program 
and student needs 


24 


2.88 


.68 


Use of technology to support science and math education 


24 


2.29 


1.00 


Students’, teachers’, and community members’ shared 
understanding of expected student outcomes in science and math 


25 


2.20 


.76 


Community business and industry support for math and science 
education programs 


24 


1.92 


.93 


Number of elementary students engaged in challenging, high 
quality science and math learning experiences 


25 


2.52 


.96 


Percentage of students taking Advanced Placement college 
examinations 


21 


1.67 


.80 


High school enrollment in math and science courses 


21 


2.00 


.71 


Minority students’ scores on science and math standardized tests 


20 


1.85 


1.76 


Middle ability students’ achievements in science and math 


24 


2.42 


2.12 


Alignment of math assessments with standards 


26 


3.58 


1.81 


Implementation of standards-based science curriculum in 
classrooms 


26 


3.27 


1.46 


Changing math activities to improve curriculum 


25 


3.52 


2.24 


Emphasis on strengthening math, science, and technology 


26 


3.31 


1.44 


Financial and administrative support for ongoing professional 
development of teachers 


26 


3.04 


1.48 


Procedures for developing and modifying curriculum and 
instructional objectives 


26 


2.92 


1.47 


Allocation of local funds to support science and math education 


26 


2.58 


1.63 


Students’ computer use in the classroom 


26 


2.88 


2.05 


Teacher-to-student ratio in science and math classes 


25 


2.04 


2.23 


Community support for math and science education programs 


26 


2.73 


2.46 


Relevant stakeholders’ embrace of goals for improving 


26 


2.85 


1.95 
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Indicators 


N 


Mean 


SD 


achievement of all students 








Number of high school students engaged in challenging, high 
quality science and math learning experiences 


25 


2.68 


2.06 


High school students’ knowledge and skills in math and science 


25 


3.12 


2.30 


Use of multiple methods to assess students and programs 


26 


2.46 


1.53 


Highest ability students’ achievements in science and math 


26 


2.58 


1.53 



Question 9— Which school in your district has had the greatest involvement with RSI? 



In the three RSIs combined, schools at the elementary and middle school/junior high levels were 
cited as having the highest RSI involvement (48 percent, Table 19). Elementary schools had a 
slightly higher involvement in UCAN and ARSI, while there was a higher involvement at the 
middle school/junior high level in Delta. In all three RSIs a lower percentage of high schools 
(36 percent) were listed as the most involved schools. These data correlate with those from 
Question 1, which show a higher percentage of elementary and middle school/junior high 
schools involved in each RSI than at the high school level. 



Table 19. Grade Level of the School with Highest RSI Involvement in Each District 



lllllllll 


Number of Schools 


Percentage at Each Level 


RSI 


Elementar 

y 


MS/JH 


HS 


Elementar 

y 


MS/JH 


HS 


Delta 


8 


10 


6 


44% 


56% 


33% 


UCAN 


14 


13 


11 


54% 


50% 


42% 


ARSI 


10 


9 


7 


44% 


39% 


30% 


Combined 


32 


32 


24 


48% 


48% 


1 1 36% 1 



Note: Percentages at each level were calculated as percentage of respondents in each RSI listing 
a school at that level. Many districts listed schools at more than one level; therefore, the 
percentages do not equal 100 percent. 

Question 10— What have been the most important factors contributing to this school’s 
involvement with the RSI? 

At the schools in the district with the highest RSI participation, the most important factors 
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contributing to involvement were support by the administration, direct contact, interest, 
professional development, and resources (see Table 20). 

Different factors were most often cited as important by districts in each of the three RSIs. 
Support and interest by the school’s principal or district were cited as important to high RSI 
involvement by 28 percent of the respondents, but was particularly important in ARSI schools 
(39 percent). Direct contact by a colleague in the school, particularly a Teacher Partner, was 
listed as important to high RSI involvement by 70 percent of the ARSI districts, as well as 
several other districts. Teacher interest and willingness to participate, as well as a positive 
attitude toward reform, were cited as important to RSI involvement by respondents in all three 
RSIs (22 percent), but were cited as important in almost a third of UCAN districts. The same 
number of UCAN districts also listed teacher professional development as important to RSI 
involvement. Respondents particularly mentioned professional development that was 
appropriate for the individual and the grade level. Resources such as money, RSI staff, 
consultant services, and materials were cited as contributing to high RSI involvement in 16 
percent of the districts. Other factors cited by at least two districts were perceived need, team 
effort, teacher stability, networking, and collaboration among schools and family involvement. 

Table 20: Factors Important to High RSI Involvement 





Number of Districts 


Percentage of Districts 


*111111 


Factor 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Overall 


Administration support 


5 


5 


9 


28% 


19% 


39% 


28% 


Direct contact 


0 


2 


16 


0% 


8% 


70% 


27% 


Teacher interest 


3 


8 


4 


17% 


31% 


17% 


22% 


Professional 

development 


3 


8 


2 


17% 


31% 


9% 


19% 


Resources 


4 


5 


2 


22% 


19% 


9% 


16% 



Question 11— What are the most significant changes in this school [school most involved in 
the RSI] as a result of RSI involvement? 

In all three RSIs, respondents reported that improved instruction was the most significant change 
(49 percent) that resulted from RSI involvement (Table 21). These instructional changes 
included more use of inquiry, hands-on activities, and activity-based instruction as well as 
changes in assessment practices. Curriculum changes were also cited as important (24 percent) 
by respondents in each RSI. These changes included curriculum alignment to standards, 
implementation of a math/science program, and a greater use of math/science equipment and 
kits. Increased communication and involvement were listed as significant changes by 21 percent 
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of the respondents. This included increased family participation, community awareness of 
math/science, and communication with the community, as well as increased teacher 
collaboration and networking with other schools. Training and education for school staff were 
listed as important changes by 19 percent of the respondents. Support for professional 
development increased, as did the quality of the programs. Several districts in each RSI reported 
improved student achievement as a significant change in those schools most involved with the 
RSI (10 percent). 

Table 21: Areas of Significant Change in School with Greatest RSI Involvement 





Number of Districts 


Percentage of Districts 




Area of Change 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Overall 


Improved instruction 


7 


11 


15 


39% 


42% 


65% 


49% 


Curriculum changes 


3 


4 


9 


17% 


15% 


39% 


24% 


Increased 

communication and 
involvement 


4 


5 


5 


22% 


19% 


22% 


21% 


Training and education 


4 


6 


3 


22% 


23% 


13% 


19% 


Increased focus on 
math/science 


1 


4 


6 


6% 


15% 


26% 


16% 


Improved student 
achievement 


3 


2 


2 


17% 


8% 


9% 


10% 



Question 12— Which RSI activities or services have contributed most to these changes [in 
the school most involved with the RSI]? 

RSI activities or services that were reported as having contributed most to changes in schools 
with high RSI involvement included professional development, RSI staff and consultants, 
curriculum-oriented activities, and resources (see Table 22). 

Respondents overwhelmingly (58 percent) cited professional development and training as the 
RSI activity or service that contributed most to the changes in schools with the highest RSI 
involvement (see Table 22). This included professional development activities for teachers, 
Leadership Institutes, team training, and professional development for principals. Having a 
Teacher Partner in the school was listed by 65 percent of the ARSI districts as contributing to the 
highest level of school involvement. Since the Teacher Partners both received and delivered 
professional development, they can be included in the professional development category. If one 
analyzes the responses in this way, 87 percent of the ARSI districts listed professional 




20 



development as important to school changes, and 67 percent of the overall responses can be 
grouped in this category. 

Several other RSI activities and services were also cited as contributing to significant changes in 
the most involved schools. RSI staff and consultants were listed by 15 percent of the 
respondents, but were seen as most helpful in ARSI (22 percent). Curriculum-oriented activities 
(13 percent) included needs assessments; program reviews; curriculum changes; and curriculum 
development, selection, and alignment. Resources cited as contributing to changes included 
funding to purchase equipment and materials, funding and grants in general, and curriculum 
materials. Collaboration and teamwork were listed as contributing to change in 10 percent of 
districts and included collaboration and networking with other districts, the resource 
collaborative and the UCAN team. 

Table 22: RSI activities or services contributing most to significant changes in schools with 
greatest RSI involvement 





Number of Districts 


Percentage of Districts 




Activity or Service 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Overall 


Professional 

development and training 


9 


16 


14 


50% 


62% 


61% 


58% 


Teacher Partner 


0 


0 


15 


0% 


0% 


65% 




RSI staff and consultants 


1 


4 


5 


6% 


15% 


22% 


15% 


Curriculum-oriented 


2 


3 


4 


11% 


12% 


17% 


13% 


Resources 


2 


2 


4 


11% 


8% 


17% 


12% 


Collaboration and 
teamwork 


1 


2 


4 


6% 


8% 


17% 


10% 



Question 13 — Which school in your district has had the least involvement with the RSI? 

While elementary schools were listed as having among the highest RSI involvement (see 
Question 9 and Table 19), they were also the level most frequently listed as having the lowest 
involvement overall (36 percent, Table 23). The responses from UCAN lower the combined 
mean. This probably has something to do with the large number of schools widely dispersed 
over a four state area and the resources available to involve all the schools. In the individual 
RSIs, elementary schools were listed as having the lowest involvement in Delta and ARSI, while 
high schools were more frequently listed in UCAN. 
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Table 23: Schools at Each Educational Level Having the Least RSI Involvement 



lillilliPl 


Number of Schools 


Percentage at Each Level 


RSI 


Elementar 

y 


MS/JH 


HS 


Elementar 

y 


MS/JH 


HS 


DELTA 


9 


2 


7 


50% 


11% 


39% 


UCAN 


6 


5 


9 


23% 


19% 


35% 


ARSI 


9 


2 


6 


39% 


9% 


26% 


Combined 


24 


9 


22 


36% 


13% 


33% 



Note: Percentages at each level were calculated as percentage of respondents in each RSI listing 
a school at that level. Many districts listed schools at more than one level; therefore, the 
percentages do not always equal 100 percent. 

Question 14— What factors have contributed to this school’s [school least involved in the 
RSI] lower involvement with the RSI? 

Factors most often cited as contributing to lower RSI involvement included lack of interest or 
support, lack of emphasis at that level or school, and lack of time (see Table 24). 

In all three RSIs, lack of interest or support was cited most frequently (30 percent) as the factor 
contributing to lower participation by the school(s) least involved in the RSI. Lack of teacher 
interest and lack of administrative support were the most frequently listed items in this category. 
Other factors included teachers resisting change, teachers teaching out of content area, lack of 
parental involvement, Teacher Partner not allowed to leave the school for meetings, no school 
representative on the RSI Leadership Team, and lack of coordination between the RSI and the 
school district. 

The next most frequently cited reason for low involvement was a lack of emphasis at that 
school’s level or at that school (16 percent). Other factors were that no Teacher Partner was 
assigned at this level and a decision that this school needed less support. 

A lack of time was cited by 12 percent of the respondents overall, but was listed by 23 percent of 
the UCAN districts as a factor leading to lower involvement. This category included factors 
listed as less staff at the school, teachers tied up with after-school activities, and staff with 
multiple responsibilities. 

Although the districts in the three RSIs are located in rural areas, only 8 percent of the 
respondents listed geographic distance or isolation as a factor leading to lower RSI involvement. 
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Table 24: Factors Contributing to Lower Involvement in Schools Least Involved with the RSI 





Number of Districts 


Percentage of Districts 


11 


Factor 


Delta 


UCAN 


ARSI 


Delta 


UCAN 


ARSI 


Overall 


Lack of interest/support 


5 


9 


6 


28% 


35% 


26% 


30% 


Decreased emphasis at 
level 


4 


0 


7 


22% 


0% 


30% 


16% 


Lack of time 


1 


6 


1 


6% 


23% 


4% 


12% 


Geographic distance 


2 


2 


1 


11% 


8% 


4% 


8% 



Summary 

Elementary schools have the highest level of participation in the RSI. This finding was 
corroborated through case studies that were conducted as part of the overall study. 

Examination of the data suggests that there are two different strategies employed by RSIs for 
providing services to school districts. ARSI apparently tried to provide a very broad menu of 
services to everyone. UCAN and Delta RSIs appear to provide core services, such as 
group/individual professional development and on-site consultation, to everyone and supplement 
with other services, perhaps tailored to the needs of individual districts. 

Professional development, particularly for teachers, was an important part of all three RSI 
projects. It was listed as the most effective RSI activity, as the greatest benefit districts gained 
from RSI participation, and as the RSI activity that contributed most to significant changes in 
schools with the greatest RSI involvement. 

Curriculum-related activities were also reported to be among the most effective RSI activities. 
Curriculum changes as a result of these activities were seen as a great benefit in UCAN and 
ARSI. However, only in ARSI were curriculum changes perceived to be among the most 
significant changes in the schools with high RSI involvement. Professional development was 
cited as an important activity that led to curriculum changes. 

A factor that facilitated education reform across all three RSIs was administrative support in the 
form of school boards, district administration, and school principals. Another factor that 
facilitated education reform was state education standards. The factors that were found to be 
barriers to education reform were money, lab equipment, and expectations for students. 
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Respondents said RSI participation most influenced math assessment alignment with standards 
(mean=3.12); emphasis on strengthening science, math, and technology (3.08); and curriculum 
implementation (3.04). RSI least influenced teacher/student ratios (mean=1.43); minority test 
scores (1.47); and the percentage of students taking advanced placement courses. 

Resources such as equipment/materials, consultants, and funding were another benefit of RSI 
involvement. They were listed as some of the most effective services by Delta and ARSI 
districts. Professional development was an important component of learning to use new 
equipment and materials. 

RSI activities that led to school staff going beyond the confines of their buildings were also 
listed as a benefit of UCAN and ARSI involvement. These activities led to increased 
communication with schools outside the district and with the community, as well as increased 
family involvement in the school. Delta and ARSI districts also listed these activities as among 
the most effective ones. 

While most RSI activities were not directed toward administrators, administrative support was 
important for high RSI involvement in Delta and ARSI. Direct contact by Teacher Partners was 
very effective in increasing school involvement in ARSI. In UCAN, teacher interest and 
professional development activities were cited as factors leading to high RSI involvement. 
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